Many natural scenes have wider dynamic ranges than those that can be recorded by conventional low dynamic range (LDR) imaging systems. An LDR image with small exposure time is under-exposed in the dark scene regions but captures the bright areas well. In contrast, an LDR image with large exposure time is saturated in the bright scene regions but captures the dark areas well. In other words, one LDR image is not able to represent the whole scene. A high dynamic range (HDR) image can be produced to represent the whole scene by sequentially capturing multiple differently exposed LDR images using normal cameras.
When an HDR image is synthesized for a scene by using multiple differently exposed LDR images, moving objects in the LDR images may cause ghosting artifacts in the final HDR image. This is often encountered in outdoor settings, wherein moving people, bus, clouds or trees waving, etc. may be captured as moving objects. To remove ghosting artifacts due to moving objects in the scene, the pixels of all LDR images are required to be properly classified into valid and invalid, and only valid pixels are used to generate the HDR image. Due to sensor and electronic noises as well as different exposures in the input LDR images, it is challenging to detect moving objects in the LDR images.
One approach is based on camera response functions (CRF) to remove ghosting artifacts. The predicted value for a pixel in an image can be computed by using the CRFs, the co-located pixel value in its reference image and their exposure times. The pixel is marked as valid if it is well approximated by the predicted value. Otherwise, it is marked as invalid in an error map. The schemes based on CRF are, however, sensitive to the estimate error of CRFs.
Another approach is based on the feature that local entropy is usually not changed much with respect to the exposure times. The local entropy is computed, and all pixels with local entropy variation larger than a threshold may be marked as invalid in an error map. Although the local entropy based method is not sensitive to the estimation error of CRFs, it is not suitable for a situation when two image regions share the same structure but with different intensity.
In another aspect, the invalid regions are usually patched by only using pixels from one single LDR image. Since moving objects usually belong to invalid regions, the dynamic ranges of moving objects in the synthesized HDR image are reduced when their dynamic ranges are inherently high.
It is thus desired to provide a movement detection scheme which is robust with respect to different exposures and variation among pixels caused by sensor and electronic noises, for the synthesis of an HDR image via a set of differently exposed LDR images.
It is also desired to provide a method to better preserve the dynamic range of moving objects in the synthesized HDR image.